Electronic device for processing image-data, for simulating the behaviour of a deformable object

ABSTRACT

An electronic device for processing image data, particularly image data pertaining to medical procedures, includes a user interface with force feedback ( 4 ) corresponding to tool reactions, a “collision” module ( 18 ) for estimating a point of intersection between a straight line embodying a displacement derived from the action of the tool and a surface mesh of a given object, and an internal forces module ( 16 ) which estimates internal forces exerted on nodes of a first part of at least a volume mesh of the object, on the basis of a displacement applied on nodes pertaining to the surface mesh containing a point of intersection, of boundary conditions, and of node tensors and link tensors, from matrices of rigidity, and a reaction module ( 20 ) for determining the reaction force of the object corresponding to its deformation estimated on the basis of the internal forces, such that the force generated by the user interface ( 4 ) is balanced by reaction force.

BACKGROUND OF THE INVENTION

The invention relates to the field of the processing of digital imagedata from a set representative of a three-dimensional (3D) image, forsimulating the deformable behavior of an object.

The invention applies more particularly, but nonexclusively, to theprocessing of a set of image data from a so-called medical image.

In numerous fields, it is very beneficial to be able to simulateinterventions by an operator, with the aid of one or more known tools,on one or more deformable objects. Here, the term intervention isunderstood to mean either a manipulation, with a view for example to adisplacement, or a local transformation, such as for example, in thecase of a surgical intervention, incision or extraction of a part of anorgan.

Simulation consists in displaying the image of an object and possibly ofthe region in which it customarily lies, and the representation of onetool at least whose <<virtual>> (in this document, words which appearwithin double-arrowhead brackets reflect the fact that a concept, in thecontext of this document, is designated with such words) displacement,relative to the object, is defined by a user interface of which aharness is maneuvered by an operator, with a view to simulating thehandling of the said tool. In order to be able to simulate the reactionof the object on the tool, the user interface is capable of generating aforce feedback, in accordance with the reactions of the tool. The termreaction force of an object is understood to mean force feedback.

In known devices, a reaction module makes it possible to determine thisreaction force of the object on the basis of an estimated deformation ofthis object. This deformation is obtained with the aid of an internalforces module and of an image refresh module. The internal forces moduleis capable, on designation of a 3D. object appearing in a set of imagedata, of establishing a field of internal forces, which isrepresentative of the deformation of the object, between nodes of avolume meshing dependent on a surface meshing of this object, on thebasis of a deformation law and of an action defined by the userinterface and representative of a maneuver of the tool.

The refresh module then makes it possible to calculate new image data ofthe object, in the presence of the estimated deformations supplementedwith the representation of the tool. These new image data which form thenew image of the object and possibly that of the region which surroundsit, are then displayed on a display device so that the operator can seein real time the result of the manipulation of the harness whichsimulates the action on the tool.

Such a device must allow the training of an operator or else thetailoring of new techniques of intervention on the object. In a fieldsuch as surgery, and more particularly still in the field oflaparoscopic surgery, this type of device may make it possible to savehuman lives. To do this, it is imperative that the simulation makes itpossible to reproduce the operator's gesture (or in other words hisaction on a tool, here virtual) as faithfully as possible. This requiresreal-time processing of the image data, coupled with reconstruction ofthe forces induced by the object in response to the deformationgenerated by the <<tool>>.

Now, on account of the calculation techniques used by known devices,estimation of the internal forces requires considerable calculationtimes which are incompatible with continuous dynamic simulation. Inother words, contemporary devices do not make it possible to display, ina manner which is continuous in respect of a human eye, the entireaction of a tool on a deformable object.

Moreover, no contemporary device makes it possible to simulate in realtime an action such as incision, or tearing, or the removal of materialfrom a deformable object.

SUMMARY OF THE INVENTION

The aim of the present invention is therefore to solve all or some ofthe aforesaid drawbacks in the field of the processing of digital imagedata of a 3D object.

It therefore proposes an electronic device for processing image data ofthe type described in the introduction, in which, on the one hand, thereis provision for a <<collision>> module capable of estimating a point ofintersection between a straight line embodying a displacement derivedfrom the defined action and the surface meshing, and on the other hand,the internal forces module is devised so as to estimate the internalforce exerted on each node of a first part at least of the volumemeshing of the object on the basis of the displacement derived from theaction, and applied to the nodes belonging to the surface mesh cellcontaining the point of intersection, of boundary conditions, and ofnode tensors and link tensors emanating respectively for each node andeach link of at least the first part at least, from stiffness matricesspecific to each volume mesh cell of at least the first part anddependent on the deformation law.

Of course, the first part of the volume mesh cell to which the abovetechnique is applied, which will subsequently be referred to as<<masses/tensors>>, can be equal to the complete volume mesh cell. Inthe contrary case (when dealing in fact with a part of this volume meshcell), the internal forces applied to the nodes of the partcomplementary to this first part (referred to for example as the secondpart) are determined on the basis of another technique, such as forexample that of finite elements relying on precalculations which arestored so as to allow real-time calculations. Such a technique is taughtin particular in the article by S. Cotin, H. Delingette, M. Bro-Nielsenand N. Ayache, <<Geometric and physical representations for a simulatorof hepatic surgery>>, published in the proceedings of the conferenceMedicine meets with virtual reality, January 1996. In what follows, thedual technique of calculating internal forces and the deformation of theobject will be referred to as a hybrid model.

This so-called masses/tensors technique used for calculating theinternal forces and the deformation of the volume mesh cell of theobject permits continuous simulation of an action exerted on a virtualtool at least. It is clear that the smaller the dimension of the firstpart of the volume mesh cell, the less will be the calculation time.

According to another characteristic of the invention, the device cancomprise a meshing module allowing it to designate the 3D object(s) onwhich the simulation is to be performed by determination of an externalenvelope, then to decompose this or these envelopes into surface meshcells, preferably of triangular form, and lastly to decompose theinternal volume of each envelope into volume mesh cells on the basis ofthe corresponding surface meshing so as to provide the volume meshing ofthe associated object. It is clear that in the preferred case of atriangular surface meshing the volume mesh cells will be tetrahedral inshape. These shapes are currently preferred since they allow accuratemodeling of an object of complex shape. However, of course, other typesof meshing may be used.

Such external envelopes and volume mesh cells may be obtained by methodsof segmentation (for example by extracting iso-surfaces) and of theDelaunay-Voronoï type respectively. All these methods are well known tothe person skilled in the art.

In one embodiment of the device, its internal forces module is capableitself of calculating the stiffness matrices of each volume mesh cell,as well as the node tensors and link tensors.

These calculations are, as was stated earlier, performed on the basis ofa deformation law which is preferably of volume linear elastic type. Inother words, the force exerted on a node depends on the displacementsrespectively of this node and of the nodes to which it is connected,relative to their respective positions of equilibrium. Of course, othermore complex deformation laws could be used, in particular non-linearlaws.

The internal forces module could also be devised so as to determine theinternal forces exerted on some at least of the nodes of the first partof the volume meshing on the basis of the deformation law and ofauxiliary surface forces dependent on stored, chosen parameters of theobject, such as for example the texture of the object, the presence ofstructures and underlying substructures, etc.

Here, the term auxiliary surface forces should be understood to mean forexample surface tensions which, in certain situations such as anincision, will make it possible to amplify a visual effect at displaylevel.

Likewise, the internal forces module may be devised so as to estimatethe displacements of the nodes of the volume meshing (at least its firstpart) on the basis of the displacement derived from the defined actionand from external forces, in particular of gravitational force typeand/or forces of interaction between objects of one and the same region.This makes it possible to take into account, on the one hand, thepartial sagging of an object under its own weight, and on the other handthe presence of neighboring objects and of the ties which exist withthese neighboring objects.

Preferably, the estimated displacements of the nodes, other than thoseof the said surface mesh cell comprising the said point of intersection,are calculated on the basis of the internal forces by successiveintegrations with the aid of a method chosen from among at least theEuler method and the Runge-Kutta method, and more preferably still bythe so-called <<order 4>> Runge-Kutta method. Of course, other methodsof integration may be envisaged.

According to yet another characteristic of the invention, the internalforces module may be capable of simulating deformations not only ofgeometrical type, but also of incision and/or removal of material and/ortearing type.

To allow the simulation of cutting (or incision) and/or of tearing (orfracture), the internal forces module is able, after determining theestimated displacements of the nodes, to delete at least one linkbetween neighboring nodes as a function of a first criterion, then toupdate the node tensors and the link tensors as a function of thedeleted link(s), and lastly to recalculate the internal forces of thenodes of at least the first part of the volume meshing.

Preferably, the first criterion pertains to at least one parameterchosen from among at least one cue transmitted by the user interface andrelating to the type of tool maneuvered, a volume variation of thevolume mesh cell comprising the link to be deleted, and a lengthvariation of a link of the volume mesh cell comprising the link to bedeleted.

Here, the term cue is understood to mean for example an item of dataspecifying that the tool is maneuvered with a view to an incision or adestruction of material.

Likewise, to allow the simulation of the removal of material, theinternal forces module is able, after determining the estimateddisplacements of the nodes, to delete a node in the event of detectingthe deletion of all the links which join the said node to theneighboring nodes or as a function of the first criterion, then toupdate the node tensors and the link tensors as a function of the nodeand of the deleted links, and lastly to recalculate the internal forcesof the nodes of at least the first part of the volume meshing.

The tools (here virtual) capable of cutting (or of incising) and/or ofremoving material are for example scalpels, cutting forceps, or elsemechanical or electrical bistoury, or alternatively lasers.

Moreover, the internal forces module is preferably devised so as, in theevent of the deletion of a link and/or of a node and before updating thelink tensors and node tensors, to add new mutually independent nodes andnew links in such a way as to locally remesh the volume meshingsubsequent to the deletion.

When the device is devised so as to work according to the aforesaidhybrid model, its internal forces module is capable of determining theinternal forces exerted on the nodes of at least a second part of thevolume meshing on the basis of boundary conditions defined by so-calledconnection nodes placed at the interface between the first and secondparts, and of a table of deformation tensors, each tensor of which isrepresentative of the influence of an elementary displacement of eachnode of at least the second part on each other node of at least thissecond part.

The boundary conditions serving in the calculation of the internalforces of the second part are preferably defined by the internal forcescalculated for the connection nodes when calculating the internal forcesof the nodes of the first part.

In the hybrid model, the internal forces module preferably proceeds bysuccessive iterations until a position of so-called <<equilibrium>> ofthe internal forces of the connection nodes is obtained. To do this,this module is devised so as to deduce from the values of the internalforces exerted on the nodes of the second part of the volume meshing,values of displacement of the connection nodes in such a way as toprovide boundary conditions which in turn make it possible to calculatethe internal forces of the nodes of the first part.

The subdivision of the volume meshing into parts (at least two) isdetermined on the basis of a predetermined criterion pertaining at leastto a parameter of the image data of the object chosen from amongphysical parameters and anatomical parameters, in particular to anintensity. This subdivision can be performed during the formation of thevolume meshing of the 3D object. Accordingly, when the device does notcomprise a meshing module, the subdivision can be performed by anexternal processing. The device can comprise a partitioning module, forexample forming part of the meshing module if the latter exists,intended to provide the subdivision of the volume meshing.

Preferably, the <<collision>> determination module is devised so as todetermine a collision between at least two tools managed by the userinterface. This makes it possible to manage conflicts when one or moreoperators maneuver at least two tools at the same time.

For this purpose, each tool is represented by at least one pointembodying its end interacting with the object, and a multiplicity ofpoints joined to one another as well as to the end, by segments,embodying its <<shank>>.

Preferably, the collision determination module estimates the coordinatesof the point of intersection between the <<tool>> and a surface meshcell as follows:

Firstly, it creates a three-dimensional space encompassing the externalenvelope of the object, then it decomposes this space into volumeblocks, the number of which is chosen so that each block comprises anumber of node of the volume meshing of the object substantially equalto the number of nodes contained in the other volume blocks, each blockintersecting the external surface comprising at least one node, andlastly it stores in multiplets the coordinates of each node withreference to the volume block which encompasses it. The presence of apoint of the tool in the space is then effected, preferably, through acomparison between the multiplets and the coordinates of the point. Oneof these multiplets makes it possible to designate the volume block ofthe space in which the point lies.

Then, preferably, the collision detection module determines the(Euclidean) distance which separates the point of the tool from thenode(s) encompassed in the designated volume block so as to determinethe smallest of these distances, termed the minimum distance. Thereafterit determines the distance (for example Euclidean) which separates thepoint from the node(s) encompassed in a predetermined number of volumeblocks neighboring the volume block in which it lies so as to compareits distances with the minimum distance. Then, it determines thecollection of surface mesh cells adjacent to the node associated withthe minimum distance so as to determine whether a segment defined by theposition of the point of the tool and by its previous positionintersects one at least of these adjacent surface mesh cells. Lastly,from this it deduces the (barycentric) coordinates of the point ofintersection (or of collision) between the object and the tool, with aview to their transmission to the internal forces module.

Preferably, the collision detection module is capable of modifying thecontents of the multiplets between two determinations of presence ofpoints of the tool inside the volume blocks, in the event of detectionof a deformation of the mesh by the internal forces module. This makesit possible to improve the accuracy of collision detection and hence ofthe calculation of the deformation.

In order to allow the most realistic possible simulation of action onone or more tools, the user interface comprises a harness maneuverableby at least one operator hand so as to simulate the maneuvering of eachtool. The operator thus maneuvers the harness, which may be a<<joystick>> possibly fitted with actuator(s) or else an articulated<<glove>> fixed on his hand, then the user interface defines thedisplacement of the associated virtual tool on the basis of thismaneuver.

According to yet another characteristic of the invention, the devicecomprises display means making it possible to display in real time (andcontinuously) the image (formed from the image data) of the object andof a representation of the tool.

The invention applies most particularly to the sets of image datarepresenting a three-dimensional digital image of a region comprising atleast one 3D object, including the designated object, and moreparticularly still to the sets of image data representing athree-dimensional digital image of a region of a living being (animal orhuman) comprising deformable anatomical structures such as the liver,the kidney, the gall bladder, or alternatively the heart.

The invention also proposes a process for processing the digital imagedata for implementing the device described above, comprising thefollowing known steps:

provide a user interface capable of generating a force feedback, inaccordance with the reactions of a tool,

establish, on the basis of a deformation law and of an action defined bythe user interface and representative of a maneuver of the tool, a fieldof internal forces between nodes of a volume meshing dependent on asurface meshing of a 3D object appearing in a set of image data,

determine the reaction force of the object which corresponds to itsdeformation estimated on the basis of the internal forces, so that theforce generated by the user interface is substantially balanced by thisreaction force,

calculate new image data of the object, in the presence of the estimateddeformations supplemented with the representation of the said tool,

and characterized in that there is provision for a step in which a pointof intersection between a straight line embodying a displacement derivedfrom the defined action and the surface meshing is estimated, and inthat the internal force exerted on the nodes of a first part at least ofthe volume meshing of the object is estimated, on the basis of thedisplacement applied to the nodes belonging to the surface mesh cellcontaining the point of intersection, of boundary conditions, and ofnode tensors and link tensors emanating respectively for each node andeach link of this part at least, from stiffness matrices specific toeach volume mesh cell of at least the first part and dependent on thedeformation law.

BRIEF DESCRIPTION OF THE DRAWINGS

Other characteristics and advantages of the invention will becomeapparent on examining the detailed description which follows, as well asthe appended drawings in which:

FIG. 1 is a perspective view of part of a simulation device according tothe invention;

FIG. 2 is a functional diagram illustrating the architecture of asimulation device according to the invention in an embodiment with twoprocessing modules;

FIG. 3 diagrammatically illustrates the mode of displacing a toolcontrolled by a type of user interface of the device according to theinvention;

FIG. 4 illustrates the external envelope (or external surface) of ahuman liver, obtained on the basis of a segmentation technique;

FIG. 5 illustrates the external envelope of the liver of FIG. 4,furnished with a surface mesh of triangular type;

FIG. 6 is a diagram illustrating a tetrahedral volume mesh obtained froma triangular surface mesh of the type of that of FIG. 5;

FIG. 7 is a diagram illustrating the displacements at an instant t oftwo nodes with respect to their respective positions of equilibrium andof the positions of equilibrium of their neighboring nodes;

FIG. 8 is a diagram illustrating a tetrahedral volume mesh cell and thedesignation of the node tensors and of the link tensors respectivelyassociated with its nodes and links;

FIG. 9 is an example of a stiffness matrix of a 3D object represented by15 nodes, the crosses (X) signifying non-zero values and the dots (.)zero values;

FIG. 10 is a mechanism illustrating the steps for determining a finalmesh of a 3D object at equilibrium;

FIG. 11 is a mechanism illustrating the steps for determining adeformation and the image of the deformed object;

FIG. 12 is a diagram illustrating the modification of the topology of atetrahedral volume mesh following the rupturing of a link;

FIGS. 13A to 13D illustrate a simulation over four successive images ofthe action of the end of a virtual tool on a liver lobe;

FIG. 14 is a diagram illustrating a decomposition of a triangularsurface mesh into two domains (or parts), in a hybrid type processing;

FIG. 15 is a mechanism illustrating a loop for determining positions ofequilibrium in a hybrid mesh;

FIG. 16 is a diagram illustrating a space encompassing the liver of FIG.4 and decomposed into parallelepipedal volume blocks;

FIG. 17 is a diagram illustrating part of a tetrahedral volume meshcontained in a parallelepipedal volume block, as well as a point ofcollision between a tool and one of the triangular surface mesh cells ofthe part of the said volume mesh; and

FIG. 18 is a diagram illustrating the embodying of a tool in the form ofpoints and segments, with reference to a part of a triangular surfacemesh.

The drawings are, in essence, of definite character. Accordingly, theyform an integral part of the present description. They may thereforeserve not only to provide a better understanding of the invention, butalso contribute to the definition of the latter.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In what follows, reference will be made to a processing of a set ofdigital image data forming a medical three-dimensional (3D) image, andmore particularly, but solely by way of example, to images of regions ofthe liver of the type of that illustrated partially in FIGS. 1 and 4,which have been obtained in a human subject.

It is however clear that the invention is not limited to the processingof medical images, and still less to that of images of the liver. Itapplies generally to the processing of digital images ofthree-dimensional (3D) objects, in particular deformable objects, with aview to simulation of their deformation by at least one virtual tool.The word deformation is understood here within its most generalacceptance, namely both of the geometrical (surface or volume) type andalso of the incision, or tearing, or alternatively removal of materialtype.

In the medical field, a set of image data forming a three-dimensional(3D) image is obtained with the aid of apparatus such as X-ray scannersor nuclear magnetic resonance apparatus (MRI), or more generally anytype of apparatus capable of acquiring variable-intensity images. Eachelementary part of a region represented by a 3D image is defined bythree spatial coordinates and at least one measured physical quantity,such as for example an intensity.

In the case of MRI, the 3D image of an observed region consists of amultiplicity of stacked 2D sections, within which the intensityvariations represent the proton density of the tissues.

Illustrated partially in FIG. 1 is a device according to the invention,intended for simulating the dynamic behavior of a three-dimensional (3D)object. Here, as indicated earlier, the 3D object is a human liver.

In the example illustrated, the device comprises a control andcalculating unit 1 coupled to a graphics station 2 comprising displaymeans, such as for example a monitor 3, as well as to a user interface 4having force feedback.

In this example, the control unit 1 and the graphics station 2 are twoseparate computers, but it is clear that they could form a singlecalculating unit.

In the embodiment illustrated, the control unit 1 is preferably a PCtype microcomputer, whereas the graphics station 2 is preferably a DECALPHA 233 MHz type workstation equipped with a 3D graphics acceleratorcard. Here it is preferable to use the most powerful calculating unit tomanage the displaying of images, owing to the need to calculate thedeformation and to display in real time both the image of the liver andof the representation of a tool, or several tools, this requiring therefreshing of the images at a video type frequency of greater than 20Hz.

Moreover, the link between the control unit 1 and the force feedbackuser interface 4 will preferably be an ISA bus, which permits datatransfer rates of 10 kHz which are well above the frequency required forthe force feedback, which generally lies between 300 and 500 Hz.

The effort feedback user interface 4 comprises a manual control 5 (orharness) of the <<joystick>> type capable of reproducing the behavior ofa minimally invasive surgical tool. It could in particular be a tool forlaparoscopic or coelioscopic surgery. In what follows, the joystick willbe regarded as simulating the movement of microscopic cutting forcepsintended to be introduced into the body of a patient via a fixed pointtermed a trocar with a view to an intervention on the liver. Of course,the invention is not limited to this type of surgery or tool alone.

The effort feedback user interface 4 could for example be the<<laparoscopic impulse engine>> (LIE) from the company ImmersionCorporation, or else the PHANToM device developed by the MassachussetsInstitute of Technology. The <<laparoscopic impulse engine>> (LIE)possesses five degrees of freedom, three to represent the displacementof the tool shank in space, a fourth for the rotation of this shankabout an axis passing through the trocar, and a fifth associated withthe opening or closing of the forceps located at the end of the shank.The effort feedback can be applied to three of these degrees of freedomwith the aid of motors. The forces are transmitted via a system ofcables and capstans thus delivering a high and progressive torque.

Illustrated diagrammatically in FIG. 3 is the mode of displacing theshank 6 of the cutting forceps controlled by the LIE. The end 7 of theshank 6 is embodied here by a sphere which diagrammatically representsthe active end (for example the cutting end) of the virtual tool. Thetrocar is embodied by a fixed point 8 through which the shank 6 of thetool passes, which shank is connected to two motors 9 and 10 which allowit displacements over a spherical cap whose radius varies according tothe length of the shank 6, which is managed by a third motor (notrepresented). The position of the end 7 of the tool on the spherical capis determined by the values of the angles α and β.

When, with the aid of his hand 11 (see FIG. 1), an operator maneuversthe grippable part of the joystick 5, he can move the axis 6 in such away as to vary the length which separates the fixed point 8 (or trocar)from the cutting end 7 thus making it possible to simulate theapproaching of the tool towards the 3D object, but he can also move thisend 7 over a spherical cap whose radius depends on the trocar/enddistance.

Of course, the user interface 4 can comprise two LIEs so as to allow thesimulation of the simultaneous handling of two tools.

A purpose of the device according to the invention is, as has alreadybeen stated, to process, in real time, digital image data formingsuccessive three-dimensional images of a 3D object (here a human liver),which are obtained, for example, by X-ray scanning, or by MRI, dependingon the object or organ concerned.

It is firstly necessary for the device to possess the set of image dataforming the image of the 3D object, the external envelope of thisobject, the surface mesh of this envelope, and the primary volume meshof the object formed on the basis of the surface mesh. All these datamay be obtained outside of the device according to the invention. Theycould in particular originate from an image data bank coupled to ameshing module. However, the device can comprise a meshing modulecapable of determining the external envelope and the surface and primaryvolume meshes on the basis of an image data set provided by an externalmeans.

The external envelope 12 (see FIG. 4) could be obtained by any techniqueknown to the person skilled in the art, and for example with the aid ofa segmentation technique of the iso-surface extraction type, such as theso-called <<3D snakes>> or the <<marching cubes>> techniques which arewell known to the person skilled in the art.

In the case of a meshing module built into the device, the latter willpreferably be housed in the graphics station 2.

The general shape of the surface mesh cells 13 into which the externalenvelope is decomposed is preferably triangular (see FIG. 5). Acriterion for optimizing the surface area of the surface mesh cellscould be implemented so as to obtain a regular mesh. Of course, othertypes of surface meshes may be used, such as in particular the<<simplex>> meshes developed by H. Delingette and disclosed in INRIAtechnical report No. 2214 entitled <<Simplex meshes: a generalrepresentation for 3D shape reconstruction>>, 1994.

A volume mesh of the 3D object, here the liver, is determined from thesurface mesh cells 13. It is clear that the general shape of the meshcells depends on the shape of the surface mesh cells 13. In this examplewhere the surface mesh cells 13 are triangular, the volume mesh cells(see FIG. 6) are advantageously tetrahedra 14 at the four vertices ofwhich are located nodes 15 over which the object's mass, which isotherwise known, is distributed.

This decomposition of the volume encompassed by the external envelope 12into mesh cells, here tetrahedra, can be obtained with the aid of aDelaunay-Voronoï type algorithm using, for example, the Simail modulemarketed by the French company SIMULOG S.A. Of course, other algorithmscould be used with a view to obtaining the volume mesh.

As was indicated earlier, the device according to the invention does notnecessarily comprise a meshing module intended for determining theexternal envelope of the 3D object and/or the decomposition into surfacemesh cells and/or the decomposition into volume mesh cells. All or someof this processing of the image data set of the 3D object can beperformed in a manner totally separate of the device according to theinvention.

Knowing the set of image data of the 3D object, and its decompositioninto volume mesh cells 14, here tetrahedral, the device will determinethe so-called <<internal>> forces which are exerted on each node 15 ofthe volume mesh. It is clear that the sum of the internal forces exertedon a given node is different from zero only when this node is shiftedfrom its equilibrium position. To do this, an external force must beapplied to at least one of the nodes, for example by a tool.

The internal forces may be determined, either over the entire volumemesh with the aid of a first method using a single so-called<<masses/tensors>> technique which will be described below, or with theaid at least of a second method using the first masses/tensors techniqueas well as a second, different technique, each technique being appliedto at least one domain arising from a decomposition of the volume mesh.This second so-called <<hybrid>> method will be described later, withreference to FIGS. 14 and 15.

Nevertheless, regardless of the method, the determination of theinternal forces is performed by virtue of an internal forces module 16(see FIG. 2) which is preferably built into the graphics station 2 (butit could be built into the control unit 1), and which requires theassistance of a collision detection module 18 built, preferably, intothe control unit 1.

The purpose of such a module is to accurately determine the locus ofcollision (or of intersection) between a point embodying a part of thetool and the external envelope of the 3D object, or more precisely asurface mesh cell of this envelope (see FIGS. 16 to 18). This makes itpossible to provide the internal forces module 16 with the designationof at least one surface mesh cell node on which the tool acts, and whichwill make it possible to determine the internal forces exerted at eachother node of the volume mesh cell.

The level of accuracy of the point of intersection (or of collision) canbe managed on the basis of the number of points used to embody, on theone hand, the end 7 of the tool, and on the other hand, the latter'sshank 6. In other words, the collision detection performed by thecollision detection module 18 can be applied at one or more pointsembodying the tool. An example of discretizing the shape of the tool(shank and end) is given in FIG. 18, in which each point other than thepoints embodying the nodes of a surface mesh cell 13 are joined togetherby segments so as to reconstruct the shape of the tool and moreparticularly that of its shank and of its end, here cutting forceps. Itis possible to apply the collision detection to the shank 6 of the toolso as to determine a collision between two shanks of simultaneouslyactuated tools.

The collision detection module 18 operates as described below.Initially, the tool is discretized into a certain number of pointsp_(k), which depends both on the instrument used and on the accuracydesired. Next, a space is defined which encompasses the 3D objectconcerned. This space is then subdivided into volume blocks, forexample, of parallelepipedal shapes. The number of volume blocks in eachdirection in space may be parametrizable, and is preferably chosen as afunction of the geometry of the 3D object. Preferably, theparametrization is chosen so that the number of nodes of the volume meshof the 3D object is substantially identical in all the volume blocks ofthe space defined by the collision detection module and containing theobject. Likewise preferably, a constraint is imposed which consists ininsisting that each volume block intersecting the external surface ofthe object encompass at least one node of the mesh.

On the basis of this decomposition of the space which encompasses the 3Dobject into volume blocks, a table is generated in which are storedmultiplets comprising the coordinates of each node of the volume meshwith reference to the definition of the volume block which encompassesit. The determination of a point of collision at the level of a surfacemesh cell 13 of the external envelope of the 3D object is performed withthe aid of a comparison between the position of a moving point p_(k) ofthe tool, which position is deduced from the information provided by theforce feedback user interface 4, as well as on the basis of themultiplets stored in the table.

To do this, a code function is defined which makes it possible tocalculate an entry inside the table and to find the list of nodes of thevolume mesh which correspond to this entry, the said nodes thenbelonging to the same volume block as the moving point p_(k). Such acode function is given by the formulae referenced (1) in the appendix,by way of example.

Then, for each node N_(i) situated in the same volume block as themoving point p_(k), the Euclidean distance between p_(k) and each of theN_(i) is determined. The smallest of the distances thus calculated isthen denoted d_(min). Next, the distance d, which separates the movingpoint p_(k) from each node N_(j) situated in a predetermined number ofneighboring volume blocks, for example 26, is calculated. If one ofthese distances d, associated with a node N, is less than d_(min), thend_(min)=d. It is clear that the larger the number of nodes inside avolume block, the longer the calculation time will be.

Next, the number of surface mesh cells 13 (here triangles) which areadjacent to the node N associated with the minimum distance d_(min)=d isdetermined. Then, for each surface mesh cell adjacent to N, a check ismade as to whether it is intersected by the segment [P_(t), P_(t−1)]defined by the positions of the moving point p_(k) at the instants t andt−1, as illustrated in FIG. 17. Thus, regardless of the rate ofdisplacement of the tool, it will always be possible to determine thepoint of intersection (or of collision), given that only the length ofthe segment [P_(t), P_(t−1)] varies as a function of the rate.

The position of the point of collision is sub-triangular. Consequently,the coordinates of this point of collision are determined by abarycentric method. These barycentric coordinates are then transmittedto the internal forces module 16 so that the force exerted by the toolon the volume mesh is estimated accurately.

Knowing the surface mesh cell intersected, the internal forces module 16deduces therefrom the nodes on which the action of the tool is exerted.Then, from the barycentric coordinates, it deduces the distribution ofthe displacement of the virtual tool over each of the nodes of thedesignated surface mesh cell. At the same time, the user interface 4provides the internal forces module 16 with the information relating tothe vector displacement of the tool, either directly via a bus, or viathe control unit 1.

The device according to the invention uses a technique referred to as<<masses/tensors>> in which the nodes belonging to the surface meshcell, the locus of the collision with the tool, are subjected to thevector displacement provided (distributed according to the barycentriccoordinates of the said nodes), then the internal forces applied at eachnode of the volume mesh are estimated. This estimation is preferablyiterative, each iteration making it possible to calculate the internalforces applied at a first <<level>> of nodes, and those of a next<<level>> made up of the nodes neighboring the nodes of this first<<level>>, and so on and so forth until all the nodes have beenprocessed. This constitutes a kind of node-to-node <<constraintpropagation>> mechanism. In fact, during the first iteration, only theinternal forces of a first node level are non-zero, those of the othernodes being zero. It is in the course of the succeeding iterations thatprogressively (<<level>> after <<level>>) the (non-zero) internal forcesof the other nodes of the mesh are obtained.

This technique makes it possible to estimate the internal forcesindependently of the direction of the links. Thus, the deformationinduced by the virtual tool on the 3D object is independent of itsvolume mesh. Such a characteristic of independence relative to theinitial mesh is particularly beneficial in the field of simulation,especially of surgical intervention, where the deformations induced by atool are generally volumic as in the case of an incision, or of a tear,or alternatively of a removal of material.

In an approach where the deformation law is of volume linear elastictype, the internal force exerted on a node N_(i) subjected to adisplacement u is given by the formula referenced (2) in the appendix.

This formula (2) can be decomposed into two parts, or two forces, oneassociated with the actual displacement of the node (or mass) N_(i),with respect to its equilibrium position, the other associated with thedisplacement of a neighboring node N₃. Let ν be the collection of nodesN_(j) neighboring node N_(i), formula 2 for the internal forces appliedto node N_(i) can then be rewritten as indicated in the formula (3)given in the appendix.

In this formula (3), the expressions between square brackets, [T_(ii)]and [T_(ij)], represent matrices of dimension 3×3 which willsubsequently be referred to respectively as node tensors and linktensors, on account of their being defined similarly to force tensorswhich are well known to the person skilled in the art.

To determine these node and link tensors, it is necessary, firstly, toevaluate the elementary stiffness matrix [K^(e)] associated with eachvolume mesh cell (here a tetrahedron) adjacent to each link arising fromnode N_(i).

As is represented in FIG. 7, the vector displacement of a node N_(i)between an instant indexed 0 and the instant t may be denoted Δx_(i).This displacement comprises three components along the three directionsin space.

A stiffness matrix represents, in respect of a 3D object, in particulara deformable one, the various relations that exist between itsconstituent elements. In other words, it conveys the influence of a nodeon other nodes, this being given by the deformation law. The numericalvalues of the components of a stiffness matrix consequently depend onthe elasticity parameters of the object, which are known. Thedeformation law can be stored in the internal forces module 16 of thegraphics station 2.

If T denotes a tetrahedral volume mesh cell, then the elementarystiffness matrix of this tetrahedron is given by the formula (4) givenin the appendix, by way of example.

Once the stiffness matrix [K^(e)] of each tetrahedron T has beencalculated, a node tensor T_(ij) or a link tensor T_(ij) is associatedwith each tetrahedron vertex (or each node), as well as with each of itslinks, as illustrated in FIG. 8. This association is performed on thebasis of implicit formulae given in the appendix under the reference(5).

The overall stiffness matrix [K] of the 3D object can be determined fromthe set of elementary stiffness matrices [K^(e)]. Such an overall matrixis illustrated, by way of example, in FIG. 9, in the case of a 3D objectrepresented by 15 nodes referenced from 1 to F. Here, the crossesrepresent non-zero values, whilst the dots represent zero values.

In order to simplify the calculations of the internal forces, it ispossible to limit the number of links in the volume mesh cell byfiltering the values of the components of the elementary stiffnessmatrices [K^(e)], for example by comparing them with a threshold. Thus,any value below this threshold is regarded as zero, this amounting todeleting links between two nodes. This technique for simplifying thematrices makes it possible to determine only the links which are vitalto the transmission of the constraints (displacement constraints)through the volume mesh.

Moreover, it is also possible to simplify the calculation of the linktensors by taking into account the symmetry property of the overallstiffness matrix [K]. Thus, a single, unique link tensor is sufficientto characterize each link. In other words, let l_(ij) be the linkdefined between nodes N_(i) and N_(j), the link tensor associated withthis link will be [T_(ij)] if i>j and [T_(ij)]^(t) if i<j. The exponentt denotes the transpose of the matrix concerned.

Having reached this juncture, the following are available: firstly, thecoordinates of each node N_(i), secondly, the external envelope, thesurface mesh of this external envelope and the final volume mesh of theentire 3D object, thirdly the links l_(ij) between two nodes Ni and Njof the set of tetrahedra T making up the volume mesh of the 3D object,and fourthly, the set of node tensors [T_(ii)] and link tensors [T_(ij)]associated with each node and each link of each tetrahedron T.

A data structure (or base) will then preferably be defined, making itpossible to ascertain, firstly, the index of the two nodes situated atthe ends of each link, secondly, the reference of a tetrahedron (volumemesh cell) adjacent to a triangle (surface mesh cell), thirdly, thestate of a tetrahedron (active or inactive), and fourthly, the state ofa link (active or inactive). This information could be useful for takinginto account deformations of the incision, tear or removal of materialtype, both at graphical and physical level. The various steps forobtaining all these parameters are summarized in FIG. 10.

In order to take into account auxiliary surface forces, making itpossible for example to amplify an effect felt during an incision orwhen cutting, the internal forces module 16 of the device according tothe invention can combine the surface and volume link forces describedearlier with tensile forces F_(i) ^(tension) of the type of those givenin formula (6) in the appendix. These forces may be chosen by varying l₀the unloaded length of the associated spring, as well as its stiffness,depending on the magnitude of the amplification desired. This magnitudemay be defined, for example, by the operator, or else by the userinterface 4 depending on the type of tool and/or the type of object onwhich the intervention takes place.

When such a consideration is taken into account, the formula (3) for theinternal force exerted on a node N_(i) turns into the formula (7), givenin the appendix.

Moreover, it is also possible to take account in the calculation of theinternal forces exerted on each node of the volume mesh, of one or moretypes of exterior forces due to the presence of other objects in theneighborhood of the 3D object concerned, as well as to the gravitationalforces which cause a 3D object to tend to sag under its own weight owingto gravity. Such exterior forces may for example be the weight exertedat a part of the 3D object by a neighboring object, or else the force ofattachment exerted by a tie, such as for example a ligament, between the3D object and a neighboring object.

It is clear that a force of the attachment type may be regarded as aboundary condition akin, for example, to a restoring force in a dampedspring possessing a fixed end, the other end being joined to a node of asurface mesh cell of the 3D object.

The determination of the deformation of a 3D object, or more preciselyof its volume mesh, requires the knowledge at each instant of thedisplacements of each node of this volume mesh. This real-timedetermination relies on a principle of dynamic animation which relatesthe sum of the forces applied to the deformable 3D object to itsacceleration. Here, the deformable 3D object is formed by a collectionof point masses (or nodes N_(i)) joined together by link forces. It isconsequently possible to apply the general equation of mechanics to avolume mesh in such a way as to simulate the dynamic behavior of the 3Dobject which it represents when the latter is deformed by exteriorforces generated by a tool whose displacement is defined by the forcefeedback user interface 4, in reaction to a maneuvering of the joystick5 by an operator.

At a given instant t, the force applied to a node N_(i) of mass m_(i) istherefore given by the general equation of dynamics given in formula (8)in the appendix.

In what follows, it will be assumed that all the nodes of the volumemesh possess the same mass (m_(i)=μ). Of course, it would be possible toproceed differently, each node possessing its own mass, which can bedetermined by knowing the makeup of the region which it represents. Thismakeup information may form part of the set of image data forming the 3Ddigital image of the object. However, it may also be providedseparately.

It is therefore possible to determine the acceleration a_(i) of eachmass or node N_(i), and consequently to retrieve firstly its velocityv_(i), then secondly its position x_(i+1) at an instant t+Δt byperforming integrations, for example numerical integrations. It is thiscalculation of all the positions x_(i+1) at the instant t+Δt which willprovide the displacements of each node and consequently the deformationof the volume mesh cell. To do this, the following types of method couldbe used: Euler, or Runge-Kutta of order 2, or more preferably of order4.

The Euler method is the one most commonly employed. It consists, forexample, in starting from a differential equation dx/dt=f(x,t), withinitial condition x=x₀ at the instant t=0. A time step At is thenchosen, such that T_(i)=iΔt, with i=0,1,2 . . . . Knowing the positionof a node x_(i) at the instant ti, it is then possible to deduce theposition of this same node at the instant t_(i+1)=t+Δt (see formula (9)in the appendix).

Two additions, two multiplications and an evaluation of the forces arethen necessary to obtain the position of a node N together with itsvelocity at an instant t+Δt, from its acceleration at the instant t. Thevelocity and the position of the node are given by way of example in theformulae (10) in the appendix.

The Runge-Kutta formula of order 4 is currently preferred on account inparticular of its numerical accuracy. It relies, for example, on theformulae (11) given in the appendix.

In this method, the external and internal forces at each node must beevaluated four times.

Of course, other integration techniques, in particular numerical ones,may be used.

In the left part of FIG. 11 is represented the sequencing of the stepswhich make it possible to determine the displacement of each node of thevolume mesh cell, or of the selected part thereof, from the vectordisplacement of the end 7 of the shank 6 of the virtual tool relative tothe 3D object, due to the action of the operator on the joystick 5. Thisvector displacement is provided by the force feedback user interface 4,via the ISA bus, preferably at the same time as the position of the end7.

As soon as the collision detection module 18 has determined a point ofintersection between the <<tool>> and the surface mesh, the internalforces module 16 can begin determining the internal forces exerted onthe nodes of the first part of the volume mesh.

To do this, it applies a vector displacement, equal to that provided bythe user interface 4 which defines the displacement of the tool, to eachnode N_(c) of the collision surface mesh cell detected (here there arethree owing to the triangular shape of the mesh cell). Next, itdetermines the internal force exerted on each node of the first part ofthe mesh by using formulae (3) or (7) depending on the external forceschosen by the operator and/or the user interface 4. This determinationcalls upon the node tensors and link tensors, [T_(ii)] and [T_(ij)],previously determined and stored. In fact, during the first iteration,although the internal forces applied at each node are calculated, onlythose of the nodes neighboring the <<collision>> nodes N_(c) arenon-zero, given that the displacements of the other nodes are zero atthis juncture in the calculation. From this is deduced the acceleration,the velocity and the displacement of the first-neighbor nodes, andconsequently their new positions.

These new positions of the displaced nodes are then frozen for theremainder of the calculation, and the internal forces module 16 repeatsits calculation so as to determine at each node of the first part (herethe whole mesh cell), the internal force (non-zero) which is applied toit, and consequently its displacement. The internal forces are thereforecalculated according to a model of the <<propagation of constraints>>type. The deformation induced by the vector displacement of the virtual<<tool>> on the volume mesh cell representing the 3D object is thenknown.

Between two loops for calculating the internal forces and thedisplacements due to a vector displacement of the <<tool>>, that is tosay while propagating constraints, it is particularly advantageous totake into account information transmitted by the user interface 4, ordefined by the operator, pertaining to the type of tool maneuvered andconsequently the type of intervention performed. Indeed, suchinformation may indicate to the internal forces module 16 that theaction of the tool is intended to cut, or incise, or eliminate material.Thus, after an iteration during which internal forces (non-zero) exertedon certain nodes have been determined, the internal forces module 16 candelete the links which are in the cutting plane of a tool for incisingor removing material (embodied by volume mesh cells, here tetrahedra)which lies in the zone of interaction of the virtual <<tool>>, on thebasis of the action information with which it is provided by thecollision detection module 18 and the user interface 4.

In this case, a certain number of links and/or of nodes are deleted. Theinternal forces module must then update the node tensors and the linktensors as a function of the deleted link(s) and/or node(s). This takinginto account of the aforesaid information serves as a deletioncriterion.

Virtual tools capable of cutting (or of incising) and/or of removingmaterial are for example scalpels, cutting forceps, or else mechanicalor electrical bistoury, or alternatively lasers.

Moreover, it is also particularly advantageous for the deletioncriterion to be capable, between two calculation loops, of performing atest pertaining to the integrity of the links between nodes whosedisplacements have just been determined and are henceforth frozen.

The purpose of this test is to determine whether certain displacementsare such that in reality they would have caused the rupture of one ormore links and/or the deletion of one or more nodes and hence of one ormore tetrahedra (or volume mesh cells). The test consists in comparingthe displacements with a predetermined link rupture threshold. This testcould pertain, for example, to the volume variation of the volume meshcell comprising the mesh cell to be deleted, and/or the length variationof the links of the volume mesh cell comprising the link to be deleted.

If no link is to be deleted, the internal forces module 16 goes to thenext iteration (next loop) so as to continue the calculation of theinternal forces (continue the propagation of constraints).

On the other hand, if the test indicates that at least one link is to bedestroyed, that is to say if the intensity of the displacement exceedsthe predetermined threshold, then the internal forces module 16 updatesthe node tensors [T_(ii)] and the link tensors [T_(ij)] of the volumemesh cell or of the first part of the latter, by taking account of thecalculated displacements. The value of certain node tensors, and linktensors, may then become zero. This is the case in particular when thefour links which join a node to the other four nodes of a giventetrahedron break. In this case, a node of the volume mesh cell, orseveral if necessary, must be deleted together with all the associatedlinks.

The operation does not stop at this juncture, given that, for reasonsconcerned with the equilibrium of the deformed volume mesh cell, thelatter must retain its general starting structure at the level of itssurface and volume meshes. Now, the removal of a node, or the rupturingof a link give rise to modifications to one or more mesh cells, these nolonger being, in the example illustrated, tetrahedral or triangulardepending on whether a volume mesh cell or a surface mesh cell is beingspoken of. Such a case is illustrated in FIG. 12 in which the action ofthe end 7 of the tool has given rise to the rupturing of four links in aparticular sectional plane. Each of the four links being divided intotwo, it is then necessary to add nodes, and consequently links, so as toreconstruct tetrahedral mesh cells in the incised zone. In other words,the volume mesh must be locally remeshed.

Given that the result of adding nodes is an increase in the total massof the 3D object, it is then necessary to modify the mass of all theother nodes of the volume mesh, and this may be done by a homogeneousdistribution.

Moreover, adding nodes entails adding links so that the mesh cellsretain their general shape, here triangular or tetrahedral.Consequently, the internal forces module 16 must create and/or destroyelementary stiffness matrices [K^(e)] and node and link tensors, andrecalculate the elementary stiffness matrices and the node tensors andthe link tensors previously calculated, so that the calculations of theinternal forces exerted on the nodes can be continued while takingaccount of these modifications.

It is clear that it is particularly advantageous for the collisionmodule 18 to be able to update its multiplets alongside the alterations(deformation) in the part of the volume mesh cell.

As soon as the calculation of the internal forces and displacements isregarded by the internal forces module as having terminated for thevalue provided of the vector displacement of the tool, the refreshmodule 17 proceeds to the determination of the new image data of theobject and of the embodying of the tool with a view to their display,whilst at the same time, the internal forces module recommences a newcalculation with a new vector displacement of the tool, possibly appliedto nodes other than those to which they were initially applied.

Substantially simultaneously with the determination of the new imagedata of the object and of the <<tool>>, a reaction module 20, preferablybuilt into the control unit 1, determines the reaction force of theobject, which corresponds to its deformation (displacements of thenodes) estimated on the basis of the internal forces. For each nodedisplaced by the tool, its initial position (at t=0) and its currentposition (at the instant t) are known. Moreover, the tensors associatedwith this node and the links which are associated with it are known,thus making it possible to apply formula (3) given in the appendix.

The reaction force is then transmitted to the force feedback userinterface (LIE) so that it generates on the joystick [lacuna] force (orforce feedback) which is substantially balanced by the said reactionforce. This makes it possible to transmit physical reactions of theobject to the operator, this being essential for good control of theintervention maneuver.

As emerges from the above description, the masses/tensors calculationtechnique makes it possible to simulate all types of deformations, bethey of geometrical type, of incision or cutting type, of tearing orfracturing type, or else of removal of material type.

Represented in FIG. 13 are four images of a human liver subjected to theaction of a virtual tool of the cutting forceps type. Here, the actionis a simple compression.

In order to improve the quality of the image, it is possible toenvisage, before each update of the scene displayed, repeating thecomplete calculation of the deformation of the volume mesh cell a numberof times, and consequently the tests pertaining to the node and linkdeletions, as well as those pertaining to the values of the componentsof the tensors. Thus, as long as the number of iterations is below achosen threshold, and as long as the calculation time remains below thetime required for the refreshing of the images on the display means,then the loop can be recommenced. On the other hand, if this is not thecase, the updating of the scene displayed is carried out.

All the main steps of the masses/tensors technique are illustrated inthe chart of FIG. 11.

The invention also proposes an electronic process for processing imagedata for implementing the aforesaid device, in which the following mainsteps are provided:

firstly, provide a user interface (4) capable of giving a forcefeedback, in accordance with the reactions of a tool,

then, estimate a point of intersection (or of collision) between astraight line embodying a displacement derived from the action definedby the user interface and the surface mesh,

then, establish a field of internal forces between nodes of a first partat least of a volume mesh dependent on a surface mesh of a 3D objectappearing in a set of image data, on the basis of a deformation law, forexample of volume linear elastic type, of a displacement, induced by anaction defined by the user interface and representative of a maneuveringof the tool, applied to the nodes belonging to the surface mesh cellcontaining the point of intersection, of boundary conditions, and ofnode tensors and link tensors emanating respectively for each node andeach link of this first part at least, from stiffness matrices specificto each volume mesh cell of at least the first part and dependent on thedeformation law,

next, determine the reaction force of the object which corresponds toits deformation estimated on the basis of the internal forces, so thatthe force generated by the user interface is substantially balanced bythis reaction force,

and lastly, calculate new image data of the object, in the presence ofthe estimated deformations supplemented with the representation of thetool.

In certain complex cases, where the number of image data forming thethree-dimensional digital image of a 3D object is very large, the timerequired for calculating the internal forces and the deformation of theentire volume mesh may become greater than the time for refreshing theimages on the display means (here a monitor). It follows that theapplication of the technique described above cannot be effected on theentire 3D object. This is why the device according to the invention mayapply a second so-called hybrid method to the image data, in which themasses/tensors technique will be applied only to a first part of thevolume mesh, whilst the complementary part, termed the second part, ofthis volume mesh will be processed with the aid of a second real-timecalculation technique relying for example on a finite element typeapproach calling upon stored precalculations. This second technique isdescribed in particular in the already cited article by S. Cotin, H.Delingette, M. Bro-Nielsen and N. Ayache, <<Geometric and physicalrepresentations for a simulator of hepatic surgery>>, published in theproceedings of the conference Medecine meets with virtual reality ofJanuary 1996.

In what follows, that part of the volume mesh to which themasses/tensors technique is applied will be called the first part, andthat part which is complementary to the first part, to which the finiteelement technique is applied via stored precalculations, will be calledthe second part.

In the hybrid model, the volume mesh (initial or final) is subdividedinto at least two domains. It is clear that it would be possible toenvisage a subdivision into three domains, or even more, by virtue of aprocessing, for example in parallel, of the various domains. Thissubdivision into domains may be performed by the device according to theinvention, when it comprises a special-purpose meshing module of thetype of that described earlier, but it may also be performed outside thedevice in the converse case.

Whether it is done by an external module or by a module internal to thedevice, the subdivision into domains results from a selection by anoperator or from an automatic detection resulting from a predeterminedcriterion applied to the image data of the 3D object. The criterioncould in particular be a comparison of intensity between the variousimage data or else between these image data and a threshold, for examplein the case of a tumor, or a detection of a physical or anatomicalparameter contained in the said image data of the 3D object, anddesignating complex zones in which there are substructures such as forexample tumors.

It is possible to envisage the mesh being subdivided into a multiplicityof regions, for example six or seven, and for that in which thecollision with the <<tool>> is detected to be automatically designatedas the first part, the complementary part then forming the second partof the volume mesh.

The device could comprise a specific partitioning module, different fromthe meshing module, for determining the subdivision into domains. Thismodule will preferably be built into the internal forces module 16.

In the hybrid model, once the subdivision into parts has been performed,the internal forces module 16 begins the calculation of the internalforces of the entire volume mesh of the 3D object. This calculation canbegin either with the first part of the mesh, with the aid of themasses/tensors technique, or with the second part of the mesh, with theaid of the finite element technique via precalculations (the so-called<<precalculations>> technique) which relies on two principles termedsuperposition and linearity.

The vector displacement of the tool induces a vector displacement ofcertain nodes of the volume mesh, which is regarded as a multiple of anelementary vector displacement. Then, the total displacement of eachother node i (those on which the action of the tool is not exerteddirectly) is determined by summing (superposition principle) thecontributions from the displacements of all the nodes j having undergonea displacement.

In order to implement this principle of superposition, it is necessaryto determine a pre-calculation table for n×n deformation tensors, wheren represents the total number of nodes of the mesh, or of a part thereof(for example only the nodes of the surface). To do this, an elementarydisplacement is applied at a given node, then the deformation which thisdisplacement induces on all the other nodes of the mesh is calculated.From this are deduced n first elementary deformation tensors, one foreach node. Then the two aforesaid operations are recommenced, applyingthe elementary displacement to another node of the mesh, this providingn second deformation tensors. These two operations are repeated for then nodes of the mesh, until the n×n deformation tensors are obtained andwhich are stored in the form of a so-called pre calculation table.

A source of information is then available which makes it possibleinstantaneously to ascertain the deformation induced on the mesh cell byan elementary displacement of a given node. It is then merely necessaryto determine the ratio of the vector displacement of the node to theelementary displacement, then to multiply the displacement of the nodesof the mesh stored in the table by this ratio (linearity principle).Stated otherwise, the total displacement of a node N_(i) is obtained bysumming the contributions from all the deformation tensors T′_(ij)denoted by the index i in the table (T′_(i0), T′_(i1), T′_(i2), . . . ,T′_(in)) without neglecting to multiply these contributions by theratio.

More detailed descriptive elements of the second so-calledprecalculations technique are indicated, in particular, in thepublication by S. Cotin, H. Delingette and N. Ayache, <<Real-time nonlinear elastic deformations of soft tissues for surgery simulation>>,INRIA research report, which will be made public after the presentPatent Application has been filed.

Of course, as in the masses/tensors technique, the second techniquetakes account of the boundary conditions, and other auxiliary and/orexternal forces.

Regardless of which part of the mesh the calculation of the internalforces begins with, it is necessary to take account of the<<connection>> nodes 19 (see FIG. 14) which are located at the interfacebetween the two parts, and which consequently belong to both these partsat the same time. According to the invention, the connection nodesdefine boundary conditions and consequently they are parameters of thedeformation of the complete mesh.

If we argue on the basis of beginning the determination of the internalforces on the first part (with the aid of the masses/tensors technique),but it would be possible to do the opposite, we determine the internalforces, and consequently the displacements, of all the nodes of thefirst part of the mesh, including those of the connection nodes. Thedisplacements of these nodes will then make it possible to determine,with the aid of the table of n×n precalculated and stored deformationtensors T′_(ij), according to the second technique described above, thedisplacements of all the nodes of the second part of the mesh. Thisprovides new boundary conditions (external forces) for the first part ofthe mesh, and these will be taken into account for recalculating theinternal forces applied at each node of the first part, and consequentlythe displacements of its nodes, including those of the connection nodes,this providing new boundary conditions for the second part of the meshwhich will in turn make it possible to recalculate the displacements ofthe nodes of the second part.

This procedure is repeated, preferably, until equilibrium is obtainedbetween the two parts of the mesh, as regards the forces exerted on theconnection nodes. This equilibrium can be fixed by a convergencethreshold, or else by a <<floating>> number of iterations depending onthe calculation time permitted, which must necessarily be less than thetime required for the refreshing of an image (frequency greater than 20Hz). The animation loop of the hybrid model is illustrated in FIG. 15where the meshes M1 and M2 respectively represent the first and secondparts of the volume mesh.

It is clear that the internal forces module 16 takes account of thedeletions and additions of links and/or of nodes when it uses the hybridmodel. Consequently, the node and link tensors used in the firstmasses/tensors technique are updated in real time.

As soon as the calculation of the internal forces and displacements isregarded by the internal forces module 16 as having terminated for avalue of the vector displacement of the tool, the said refresh module 17proceeds to the determination of the new image data of the object and ofthe embodying of the tool with a view to their display, whilst at thesame time, the internal forces module 16 recommences a new calculationwith a new vector displacement of the tool, possibly applied to nodesother than those to which they were initially applied.

It is clear that in the case of a hybrid model, it is particularlyadvantageous for the collision module to define subtables of multipletsfor each part of the volume mesh cell. Moreover, it is particularlyadvantageous, in a desire for accuracy when detecting the points ofcollision, for the multiplets of the subtables to be updated alongsidethe alterations (deformation) of the various parts of the volume meshcell.

The device according to the invention can be implanted in a memory, forexample a mass memory, of one or more means of calculation of the workstation and/or computer type, in the form of software modules.

For all useful purposes, it is specified that more detailed descriptiveelements are indicated in the doctoral thesis by S. Cotin, <<Modèlesanatomiques déformables en temps-réel—Application à la simulation dechirurgie avec retour d'effort>> [Real-time deformable anatomicalmodels—Application to the simulation of surgery with force feedback],submitted on Nov. 19, 1997, as well as in the already cited publication<<Real-time non linear elastic deformations of soft tissues for surgerysimulation>>, by S. Cotin, H. Delingette and N. Ayache. These documentswill be made public after the present Patent Application has been filed.

The invention is not limited to the embodiments described above, merelyby way of example, but it encompasses all the variants which could beenvisaged by the person skilled in the art within the scope of theClaims below.

Thus, a 3D image processing device, and the associated process, havebeen described, in which the action on a tool was induced by themaneuvering of a <<joystick>> by an operator, then defined by the userinterface. However, it would be possible to envisage the action on thetool being defined directly by the user interface, this action beingstored and decomposable into elementary subactions, the operator nowintervening only so as to decide on the choice of certain elementarysubactions, for example with a view to training (or learning).

Moreover, devices and the associated process have been described, whichare intended for processing images, especially medical, and inparticular of the liver. However, it is clear that the invention is notlimited to this field alone. It applies also to sectors other than themedical field, as well as to non-medical fields in which the real-timeprocessing of images of 3D objects on which a simulation ofintervention, in the widest sense of the term, is of particular benefitequally in regard to the teaching of intervention techniques as inregard to the improving or tailoring of new intervention techniques.

Additionally, processing models have been described in which thematrices and tensors were calculated on the basis of a volume linearelastic deformation law. However, other types of deformation laws may beused, and in particular laws of the non-linear type.

Lastly, a <<hybrid>> model has been described which calls upon twodifferent techniques, a first technique of real-time calculationreferred to as <<masses/tensors>>, the second technique being, in theexample chosen, a so-called <<precalculations>> real-time calculationtechnique relying on a finite element approach. However, it is clearthat any other second technique could be envisaged, whether or not itcalls upon precalculations, provided that it allows image dataprocessing compatible with a dynamic simulation of a real-timeintervention.

Appendix

code=x+y*N _(blocks x) +z*N _(blocks y) *N _(blocks x)$x = {{int}( \frac{{Px} - {b\min}_{x}}{{block}_{{width}\quad x}} )}$$y = {{int}( \frac{{Py} - {b\min}_{y}}{{block}_{{width}\quad y}} )}$$z = {{int}( \frac{{Pz} - {b\min}_{z}}{{block}_{{width}\quad z}} )}$

Formulae (1):

<<int>>: function giving the integer part of its argument,

bmin_(i) (i=x,y,z): lower bounds on the space encompassing the object,

block_(width i) (i=x,y,z): dimensions of a volume block,

N_(blocks i) (i=x,y): number of volume blocks in the x and y directionsin space.

Formula (2):

F _(i) ^(int)(t)=[M]u

u: vector of dimension 3*(N_(v)+1) representing the displacement of nodeN_(i) and of its neighbors,

N_(v): number of neighbors of node N_(i), and

[M]: deformation matrix of dimension 3*3*(NV+1) representing thedeformation law of the object (known otherwise).

Formula (3):$F_{i}^{int} = {{\lbrack T_{ii} \rbrack \Delta \quad x_{i}} + {\sum\limits_{j \in v}\quad {\lbrack T_{ij} \rbrack \Delta \quad x_{j}}}}$

Formula (4):

[K ^(e)]=Volume(T)[Γ_(T)]^(t) [E _(T)][Γ_(T)]

[E_(T)] is, for example, given by the matrix:$\lbrack E_{T} \rbrack = {\begin{matrix}{\lambda + {2\quad \mu}} & 0 & 0 & 0 & \lambda & 0 & 0 & 0 & \lambda \\0 & \mu & 0 & \mu & 0 & 0 & 0 & 0 & 0 \\0 & 0 & \mu & 0 & 0 & 0 & \mu & 0 & 0 \\0 & \mu & 0 & \mu & 0 & 0 & 0 & 0 & 0 \\\lambda & 0 & 0 & 0 & {\lambda + {2\quad \mu}} & 0 & 0 & 0 & \lambda \\0 & 0 & 0 & 0 & 0 & \mu & 0 & \mu & 0 \\0 & 0 & \mu & 0 & 0 & 0 & \mu & 0 & 0 \\0 & 0 & 0 & 0 & 0 & \mu & 0 & \mu & 0 \\\lambda & 0 & 0 & 0 & \lambda & 0 & 0 & 0 & {\lambda + {2\quad \mu}}\end{matrix}}$

λ and u: LAMÉcoefficients,

[Γ_(T)]^(t) is given, for example, by the matrix:$\lbrack T_{T} \rbrack^{t} = {\begin{matrix}{a11} & {a12} & {a22} & 0 & 0 & 0 & 0 & 0 & 0 \\{a21} & {a22} & {a23} & 0 & 0 & 0 & 0 & 0 & 0 \\{a31} & {a32} & {a33} & 0 & 0 & 0 & 0 & 0 & 0 \\{a41} & {a42} & {a43} & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & {a11} & {a12} & {a22} & 0 & 0 & 0 \\0 & 0 & 0 & {a21} & {a22} & {a23} & 0 & 0 & 0 \\0 & 0 & 0 & {a31} & {a32} & {a33} & 0 & 0 & 0 \\0 & 0 & 0 & {a41} & {a42} & {a43} & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 0 & 0 & {a11} & {a12} & {a22} \\0 & 0 & 0 & 0 & 0 & 0 & {a21} & {a22} & {a23} \\0 & 0 & 0 & 0 & 0 & 0 & {a31} & {a32} & {a33} \\0 & 0 & 0 & 0 & 0 & 0 & {a41} & {a42} & {a43}\end{matrix}}$

α_(ji) first partial derivatives of the barycentric coordinates λ_(j)(x)of a point x situated inside a tetrahedron T:${\frac{\delta \quad {\lambda_{j}(x)}}{\delta \quad x_{i}} = \alpha_{ji}},\quad {i = 1},2,{3;\quad {j = 1}},2,3,4$

Formulae (5):

a) for the force tensor [T_(ij)]:${ T_{k,l}arrow{T_{k,l} + {\sum\limits_{i = 1}^{4}\quad K_{{s_{i} + {4k}},{s_{i} + {4l}}}^{e}}} ;\quad {k = 1}},2,{3;\quad {l = 1}},2,3$

s_(i)=local (m_(i)): local index of node N_(i),

<<local>>: transfer function assigning a local index (lying between 1and 4 in the case of a tetrahedron and corresponding to one of the fourvertices of T) to node N_(i) represented here by its mass m_(i).

b) for the link tensor [T_(ij)]:${ T_{k,l}arrow{T_{k,l} + {\sum\limits_{i,{j = 1}}^{4}\quad K_{{s_{i} + {4k}},{s_{j} + {4l}}}^{e}}} ;\quad {k = 1}},2,{3;\quad {l = 1}},2,3$

s_(i)=local (m_(i)) and s_(j)=local (m_(j))

Formula (6):$ {F_{i}^{tension} = {\sum\limits_{{({i,j})} \in v}\quad \{ {{{- {K_{({i,j})}( {{l(t)} - \alpha_{0}} )}}\frac{l_{ij}}{l(t)}} + {{k_{({i,j})}^{d}( \frac{\partial l}{\partial t} )}\frac{l_{ij}}{l(t)}}} }} )$

 if i and j s

F_(i) ^(tension)=0 otherwise

α₀<l₀, where l₀ is the unloaded length of a spring whose stiffness ischosen according to the magnitude of the amplification desired.

Formula (7):$F_{i}^{int} = {{\lbrack T_{ij} \rbrack \Delta \quad x_{i}} + {\sum\limits_{j \in v}\quad {\lbrack T_{ij} \rbrack \Delta \quad x_{j}}} + F_{i}^{tension}}$

Formula (8):

F _(i)(t)=m _(i) a _(i)(t)=F _(i) ^(int)(t)+F _(i) ^(ext)(t)

Formula (9):

x _(i)+1x _(i) +Δt*f(x _(i) ,t _(i))

Formulae (10):

v _(i+1) =v _(i) +Δt*a _(i)

P _(i+1) =P _(i) +Δt*v _(i)

P_(i): position of node N at the instant t_(i).

Formulae (11):

δx ₁ =Δt·f(x _(i) ,t _(i))

δx ₂ =Δt·f(x _(i)½δx ₁ ,t _(i)+½Δt)

δx ₃ =Δt·f(x _(i)½δx ₂ ,t _(i)+½Δt)

δx ₄ =Δt·f(x _(i) +δx ₃ ,t _(i) +Δt)

x _(i+1) =x _(i)+⅙δx ₁+⅓δx ₂+⅓δx ₃+⅙δx ₄

the velocity v(t+Δt)=v_(i+1) and the position P(t+Δt)=P_(i+1) arededuced from the acceleration a_(i) through the following equations:

δv ₁ =Δt·a _(i)

δP ₁ =Δt·v _(i)

 δv ₂ =Δt·a′ _(i)

δP ₂ =Δt·v′ _(i)

δv ₃ =Δt·a″ _(i)

δP ₃ =Δt·v″ _(i)

δv ₄ =Δt·a′″ _(i)

δP ₄ =Δt·v′″ _(i)

v _(i+1) =v _(i)+⅙δv ₁+⅓δv ₂+⅓δv ₃+⅙δv ₄

P _(i+1) =P _(i)+⅙δP ₁+⅓δP ₂+⅓δP ₃+⅙δP ₄

a′_(i), a″_(i) and a′″_(i): accelerations calculated respectively fromthe <<intermediate>> positions δP₁, δP₂ and δP₃.

What is claimed is:
 1. An improved electronic device for processingimage data, of the type that includes a user interface (4) capable ofgenerating a force feedback, in accordance with the reactions of a tool(6, 7), an internal forces module (16) able, on designation of a 3Dobject appearing in a set of image data, to establish a field ofinternal forces between nodes (N) of a volume meshing dependent on asurface meshing of this object, on the basis of a deformation law and ofan action defined by the user interface (4) and representative of amaneuver of the said tool, wherein the volume meshing includes linksbetween the nodes thereof, and additionally includes volume mesh cellsthat are defined by the notes and the links, and a reaction module (20)for determining a reaction force of the object which corresponds to itsdeformation estimated on the basis of the internal forces, so that theforce generated by the user interface (4) is substantially balanced bythis reaction force, and a refresh module (17) for calculating new imagedata of the object, in the presence of its estimated deformationssupplemented with a representation of the said tool, wherein theimprovement comprises: a collision module (18) able to estimate a pointof intersection between a straight line embodying a displacement derivedfrom the said defined action and the said surface meshing, wherein theinternal forces module (16) is devised so as to estimate the internalforce exerted on each node of a first part at least of the volumemeshing of the object on the basis of the said displacement applied tothe nodes belonging to a surface mesh cell containing the said point ofintersection, of boundary conditions, and of node tensors and linktensors emanating respectively for each node and each link of this firstpart at least, from stiffness matrices specific to each volume mesh cellof at least the said first part and dependent on the deformation law. 2.Device according to claim 1, further comprising a meshing module able todesignate the said 3D object by determination of an external envelope,then to decompose the said envelope into the said surface mesh cellswith a view to the decomposing of the internal volume of the envelopeinto the said volume mesh cells.
 3. Device according to claim 2, whereinthe said external envelope is obtained by a method of segmentation. 4.Device according to claim 2, wherein the said surface mesh cells aretriangles.
 5. Device according to claim 4, wherein the said volume meshcells are tetrahedra formed from the triangular surface mesh cells. 6.Device according to claim 5, wherein the said volume mesh cells areobtained by a Delaunay-Voronoi method.
 7. Device according to claim 2,wherein the internal forces module (16) is devised so as to calculatethe stiffness matrices of each volume mesh cell, as well as the saidnode tensors and link tensors.
 8. Device according to claim 1, whereinthe said deformation law is a volume linear elastic law.
 9. Deviceaccording to claim 1, wherein the internal forces module (16) is devisedso as to calculate the internal force exerted on each node of the saidfirst part of the volume meshing on the basis of the product of its nodetensor and the estimated displacement of this node, and of a summation,over the set of neighboring nodes possessing a link with the said node,of the product of the link tensor, associated with the link between theneighboring node and the said node, and of the estimated displacement ofthis neighboring node.
 10. Device according to claim 1, wherein the saidinternal forces module (16) is able to determine the internal forcesexerted on some at least of the nodes of the first part of the volumemeshing on the basis of the deformation law and of auxiliary surfaceforces dependent on stored, chosen parameters of the said object. 11.Device according to claim 1, wherein the said internal forces module(16) is able to estimate the displacements of the nodes of the volumemeshing on the basis of the displacement derived from the defined actionand from at least one external force.
 12. Device according to claim 1,wherein the said estimated displacements of the nodes, other than thoseof the said surface mesh cell comprising the said point of intersection,are obtained by a method chosen from among at least the Euler method andthe Runge-Kutta method.
 13. Device according to claim 12, wherein thesaid deformation is obtained by the so-called order 4 Runge-Kuttamethod.
 14. Device according to claim 1, wherein the said internalforces module (16) is able, after determining the estimateddisplacements of the nodes, to delete at least one link betweenneighboring nodes or a volume mesh cell as a function of a firstcriterion, then to update the node tensors and the link tensors as afunction of the deleted link(s), and lastly to recalculate the internalforces of the nodes of at least the first part of the volume meshing.15. Device according to claim 14, wherein the first criterion pertainsto at least one parameter chosen from among at least one cue transmittedby the said user interface (4) and relating to the type of tool (6, 7)maneuvered, a volume variation of the volume mesh cell comprising thesaid link or the said volume mesh cell to be deleted, and a lengthvariation of a link of the volume mesh cell comprising the said link tobe deleted.
 16. Device according to claim 15, wherein the said internalforces module (16) is able, after determining the estimateddisplacements of the nodes, to delete a node in the event of detectingthe deletion of all the links which join the said node to theneighboring nodes or as a function of the said first criterion, then toupdate the node tensors and the link tensors as a function of the nodeand of the deleted links, and lastly to recalculate the internal forcesof the nodes of at least the first part of the volume meshing. 17.Device according to claim 16, wherein the said internal forces module(16) is able, in the event of the deletion of a link and/or of a nodeand before updating the link tensors and node tensors, to add new nodesand new links in such a way as to remesh the said volume meshing. 18.Device according to claim 1, wherein the internal forces module (16) isdevised so as to estimate the internal force exerted on each node of thesaid volume meshing of the said object.
 19. Device according to claim 1,wherein the internal forces module (16) is able to determine theinternal forces exerted on the nodes of at least a second part of thesaid volume meshing on the basis of boundary conditions defined byso-called connection nodes placed at an interface between the first andsecond parts, and of a table of deformation tensors, each tensor ofwhich is representative of the influence of an elementary displacementof each node of at least the said second part on each other node of atleast this second part.
 20. Device according to claim 19, whereinboundary conditions serving in the calculation of the internal forces ofthe second part are defined by the internal forces calculated for thesaid connection nodes in the guise of nodes of the first part. 21.Device according to claim 20, wherein the internal forces module (16) isdevised so as to deduce from the values of the internal forces exertedon the nodes of the second part of the volume meshing, values ofdisplacement of the connection nodes in such a way as to provideboundary conditions which in turn make it possible to calculate theinternal forces of the nodes of the first part.
 22. Device according toclaim 21, wherein the internal forces module (16) is able to recalculatethe values of the internal forces exerted on the nodes of the first andsecond parts of the volume meshing on the basis of the boundaryconditions determined in succession on the connection nodes, until aposition of so-called equilibrium of the internal forces of the saidconnection nodes is obtained.
 23. Device according to claim 19, whereinthe parts of the volume meshing are determined on the basis of apredetermined criterion pertaining at least to a parameter of the imagedata of the said object chosen from among physical parameters andanatomical parameters.
 24. Device according to claim 19, wherein thesecond part is complementary to the said first part.
 25. Deviceaccording to claim 19, further comprising a partitioning module able tosubdivide the said volume meshing into the said parts, including thesaid first and 20 second parts.
 26. Device according to claim 1, whereinthe said collision module (18) is able to determine a collision betweenat least two tools managed by the said user interface (4).
 27. Deviceaccording to claim 26, wherein each tool comprises at least one end (7)which interacts with the object and is represented by at least onepoint.
 28. Device according to claim 27, wherein each tool has a shankthat is represented by a multiplicity of points joined to one another aswell as to the end(s) (7), by segments.
 29. Device according to claim27, wherein the said collision module (18) is able to create athree-dimensional space encompassing the external envelope of the saidobject, then to decompose the said space into volume blocks, the numberof which is chosen so that each block comprises a number of node of thesaid volume meshing of the object substantially equal to the number ofnodes of the other blocks, each block intersecting the said externalsurface comprising at least one node, and lastly to store in multipletsthe coordinates of each node with reference to the volume block whichencompasses it.
 30. Device according to claim 29, wherein the saidcollision module (18) is able, for each tool, to determine the presenceof a point of the tool in the said space through a comparison betweenthe said multiplets and the coordinates of the said point, then thevolume block of the said space in which the said point lies.
 31. Deviceaccording to claim 30, wherein the said collision module (18) is able,for each tool, to determine the distance which separates the said pointfrom the said node(s) encompassed in the said volume block so as todetermine the smallest of these distances, termed the minimum distance,then to determine the distance which separates the said point from thesaid node(s) encompassed in a predetermined number of volume blocksneighboring the volume block in which it lies so as to compare itsdistances with the said minimum distance, then to determine thecollection of surface mesh cells adjacent to the node associated withthe said minimum distance so as to determine whether a segment definedby the said position of the point of the tool and by its previousposition intersects one at least of these adjacent surface mesh cells,thus making it possible to estimate the barycentric coordinates of thepoint of collision between the said object and the said tool, with aview to their transmission to the said internal forces module (16). 32.Device according to claim 31, wherein the collision detection module(18) is able, for each tool, to modify the contents of the saidmultiplets between two determinations of presence of points inside thesaid volume blocks, in the event of detection of a deformation by thesaid internal forces module (16).
 33. Device according to claim 1,wherein the user interface (4) comprises a harness (5) maneuverable byat least one operator hand so as to simulate the maneuvering of the saidtool.
 34. Device according to claim 1, further comprising display means(3) coupled to the refresh module (17) for displaying in real time inimage form at least the image data of the object and of a representationof the tool.
 35. Device according to claim 1, wherein the set of imagedata represents a three-dimensional digital image of a region includingthe 3D object.
 36. Device according to claim 35, wherein the digitalimage is a medical image.
 37. An improved electronic process forprocessing image data, of the type that includes the steps of providinga user interface (4) capable of generating a force feedback, inaccordance with the reactions of a tool (6, 7), establishing, on thebasis of a deformation law and of an action defined by the userinterface (4) and representative of a maneuver of the said tool, a fieldof internal forces between nodes of a volume meshing dependent on asurface meshing of a 3D object appearing in a set of image data, whereinthe volume meshing includes links between the nodes thereof, andadditionally includes mesh cells that are defined by the nodes and thelinks, determining the reaction force of the object which corresponds toits deformation estimated on the basis of the internal forces, so thatthe force generated by the user interface (4) is substantially balancedby this reaction force, and calculating new image data of the object, inthe presence of the estimated deformations supplemented with therepresentation of the said tool, wherein the improvement comprises:estimating a point of intersection between a straight line embodying adisplacement derived from the said defined action and the said surfacemeshing is estimated, and estimating the internal force exerted on thenodes of a first part at least of the volume meshing of the object, onthe basis of the displacement applied to the nodes belonging to thesurface mesh cell containing the said point of intersection, of boundaryconditions, and of node tensors and link tensors emanating respectivelyfor each node and each link of this part at least, from stiffnessmatrices specific to each volume mesh cell of at least the said firstpart and dependent on the deformation law.
 38. Device according to claim3, wherein the method of segmentation comprises extracting iso-surfaces.39. Device according to claim 11, wherein the at least one externalforce comprises gravitational force.
 40. Device according to claim 23,wherein the parameter of the image data is intensity.
 41. Deviceaccording to claim 35, wherein the region additionally includes at leastone further 3D object.
 42. An improved method for permitting a user tomanipulate a 3D virtual object with the aid of a user interface thatsenses movement of a tool by the user and that provides force feedbackto the user, the virtual object appearing in a set of image data andhaving a surface meshing and a volume meshing, said method being of thetype that includes establishing, on the basis of a deformation law, afield of internal forces between nodes of the volume meshing when thevirtual object is deformed by the tool; determining a reaction forcethat is to be exerted by the user interface when the virtual object isdeformed by the tool, the deformation of the virtual object beingestimated at least in part on the basis of the internal forces; andcalculating new image data of the object, in the presence of theestimated deformation supplemented with a representation of the tool;wherein the improvement comprises: estimating a point of intersectionbetween a straight line embodying a displacement of the tool and thesurface meshing, wherein the internal force exerted on the nodes of apart of at least the volume meshing of the virtual object is estimatedon the basis of at least one factor, the at least one factor including adisplacement applied to the nodes belonging to a surface mesh cellcontaining the point of intersection.